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COHRED perspectives on 


The Council on Health Research for Development 
(COHRED) supports countries to optimise their health 
research potential to improve health and reduce 
health inequities and to generate economic and social 
prosperity. © COHRED prioritises the poorest 
countries/regions/populations. It achieves its mission 
by providing support (processes, methods, technical 
information, learning opportunities, | advocacy, 
motivation, seed-funds) which countries can use to 
develop their own health research systems and 
partnerships to pursue high quality research focused 
on priorities for advancement in health as identified 


by them. 


Research should take place in and help build National 
Health Research Systems (NHRS). These systems 
should ensure optimal governance of research, human 
and financial resource management, and research 
production and utilisation. Priority setting and 
developing the capability for priority setting are 
minimum requirements for a research system to 
function well. 


Since 1993, COHRED has published working papers, 
studies and a set of criteria to help guide priority 


priority setting for health research 


setting at national level, and supported some 25 
countries in developing national priority setting 


processes. 


Although many organisations in health research, 
science and technology and other sectors have 
developed methods and tools for priority setting 
in health and for research, scarce information 1s 
available on the best processes to ensure 
priority setting for national health research is 
sustainable, remains up to date, and leads to 
relevant actions. 


The COHRED Priority Setting Initiative works 
with institutions, development and_ health 
research partners to find and share practical 
solutions to priority setting. This Working Paper 
is the first step in a ‘learning spiral’ that 
engages research managers and institutions to 
share experiences. From these interactions, a 
process and approach will be distilled that 
people, institutions and other development 
partners can use or adapt to make priority- 
setting work for their countries. 
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Learning Spiral 

The COHRED Learning Spiral is COHRED’s process 
of dialogue and critical review by groups of health 
research experts — from South and North — of 
approaches to how developing countries can improve 
their health research systems. Specific learning points 
in the Spiral are opportunities for publications and 
lessons to emerge from this experience. Working 
Papers are one point in this ongoing learning process. 
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Priority setting for health research 


Toward a management process for low and middle income countries 


Executive Summary 


This Working Paper documents the 
interactions of a ‘think tank’ consultation, 
initiated by COHRED, bringing together 
health research managers from Brazil, South 
Africa, The Netherlands, The Philippines, the 
private sector, the Pan American Health 
Organization (PAHO), the Global Forum for 
Health Research and COHRED. 


Its purpose was to better understand the issues 
countries face and their needs in moving 
priority setting forward as well as seeking 
advice from professionals on how COHRED 
can support them in managing a process of 
setting and measuring progress in health 
research priorities. 


Rather than having priorities ‘reviewed’ and 
‘set’ through a workshop or national activity 
that produces a plan reflecting the situation at 
one point in time, the discussion in this think 
tank examined what process is needed so that 
national health research priorities are managed 
in a dynamic way, and are measured, updated 
and can evolve with the reality of the national, 
operational and political context. 


The key question, then, is: how to improve the 
management of a_ priority setting process to 
optimally evolve within the reality of a country’s 
research environment? 


This suggests that the process and the focus on 
management - the staff skills, roles and 
relationships to be built and managed in a 
health research system - need to take 
precedence over the tool and method used. In 
essence, there is a good chance that if the 


research manager can ‘get the process right’ 
the rest will fall into place. Tools, then, become 
an important and useful part of defining what 
needs to be delivered through the process and 
management (performance) goals, but these do 
not necessarily define the quality or the impact 
of priority setting. 


When focusing on the priority setting process, it 
is necessary to approach it as a continuous and 
cyclical activity that will involve an increasing 
number of people over time and will build on 
better and more accurate data as the process 
continues. The focus should be on action 
resulting from the defined agenda. It is 
important to show that setting a health 
research agenda will lead to action, even 
though, at the beginning, it may only be 
possible to implement part of the total agenda. 
This orientation towards action, and the notion 
that the process will improve over time, will 
help to motivate people to remain involved in 
the agenda setting. 


This publication is a step in a learning process 
where results will be shared and validated 
with a broader number of countries, in 
learning interactions throughout the year (see 
also: www.cohred.org/priority setting). It is 
expected that a number of useful country 
experiences and guidelines of processes that 
have worked - or not - will emerge from this 
learning process over the coming two years. 


This think tank did not seek to draw 
conclusions or to reach consensus. It was an 
issues identification and problem solving 
session between practitioners with the aim of 
outlining some key process areas needed for 
successful priority setting in health research. 


1. Key issues on priority setting for health research 


1.1 Introduction 


Making health research priority 
setting work for countries and 
research managers 


A special think tank consultation was held to 
engage with research managers and 
practitioners from several developed and 
developing countries to increase the 
understanding of the key issues necessary for a 
successful national priority setting process for 
health research. The first day of discussions 
took stock of existing methods and allowed 
participants to share their experiences. The 
second day looked at the way forward, 
specifically addressing three questions: how to 
implement a priority setting agenda?; how to 
monitor and evaluate the priority setting process?; 
and how to ensure that the process goes beyond the 
planning phase and remains alive? 


This consultation is the starting point of an 
ongoing learning process which will engage 
larger numbers of people and institutions from 
a wide range of countries. It aims to develop a 
dynamic approach to priority setting that is 
continuously adapted as new experiences 
become available. The approach will guide 
research managers in the setting and updating 
their priorities, and support them in using 
priority setting as a key strategy for 
strengthening the National Health Research 
System (NHRS). 


This working paper presents the country 
experiences discussed (section 1.3) and extracts 
the main issues that the research managers 
participating in the consultation consider to be 
relevant for a successful priority setting 
process (section 1.2). It also identifies activities 
and roles that COHRED can play in 
supporting countries in their priority-setting 
work (section 1.4). 


1.2 The process of priority setting: steps and important issues 


1.2.1 Scanning the environment: 
Where are the main resources for 
health research? 


Before engaging in a national priority setting 
process, it is important to consider the 
environment within which it should take 
place. Key questions include: who is involved 
in research for health? in the country? How do 
policy-makers perceive research? What is the 
capacity available to do, use and fund 
research? 


A scan of the health, research and political 
environment in the country will guide 
decisions on whether the time is right to start a 
priority setting process, or if other parts of the 
health research system first need to be 
strengthened. 


Changes in a country’s political situation can 
determine the decisions around priority 
setting. If upcoming elections are likely to 
change political leadership in the country, the 
best decision may be to start the priority 
setting process when the new political 
leadership is in place. Likewise, changes in 
administration (i.e. Health Ministry) can 
influence the timing and frequency of the 
priority setting cycle. 


A situation analysis is a barometer on the level 
of awareness of the need for research to inform 
health and health care decisions. If awareness 
is low, advocacy to build the case for health 
research generally will be needed to gain 
commitment for a priority setting process at a 
later stage. 


1.2.2 Inclusiveness: Who should be 
involved? 

Involvement of multiple stakeholders in 
priority setting is of fundamental importance 
both for the credibility of the process and to 
give the best possible chance _ for 
implementation of priorities. All direct 
stakeholders - those who own the problem 
and those that can provide the solutions - need 
to actively participate in setting priorities. 


The Essential National Health Research 
principle of involving the community, 


"8 The term health research is use 
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d here with a broad connotation in mind. Likewise, national health resear 
ee enamnace research for health outside the health sector. 


scientists and policy makers/ administrators in 
the process of priority setting has been tried in 
both developing and industrialised countries, 
to broaden ownership. Bringing together these 
disparate groups has several challenges. These 
include: how to involve members of the 
community - who may not have the ‘right’ 
expertise in the eyes of the health research and 
policy community - in a way that creates a 
meaningful exchange with research and 
political players; how to best involve the 
private sector or donors - and ensure a 
contribution or realignment of their agendas to 
national priorities; how to best link the 
technical and political sides of the debate; how 
to bring together multiple sectors, such as 
health, science and technology, agriculture and 
ensure a holistic approach to health and health 
research. 


Arguably, the most useful perspective to 
inform a priority setting process is the view of 
research managers and colleagues who have 
gone through this experience in other 
countries. This ‘community’ can provide rich 
and practical input to the process in terms of 
what has worked well or not, or what tools 
and methods have helped solve particular 
problems. Today, practical links of this kind 
are weak among professionals between 
countries. More systematic efforts are needed 
to promote the sharing of these experiences. 


1.2.3 Methods, tools, criteria: How to 
‘do’ priority setting? 


Since the mid-1990s, a number of methods for 
measuring the magnitude of health problems 
and their distribution in a country have 
emerged. In parallel with this, several models 
for defining research priorities have been 
suggested and applied. 


Common features of most of these tools and 
methods are estimations of health problems, 
identification of gaps in the knowledge about 
ways to eliminate them and of research needed 
to control them. The focus is on past and 
current health problems. Common criteria for 
the choice of priorities include the possibility 
to address the problem through research, the 
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feasibility and cost of the research and the 
potential outcome, impact and _ cost- 
effectiveness of interventions resulting from 
the research (See Section 2 Methods for Priority 
Setting and Annotated Bibliography). 


An effective priority setting approach should 
meet the objectives of the health system. It 
should aim to achieve maximum health 
benefits to the population it serves within the 
available budget and respecting specific equity 
considerations. Criteria for priority setting 
should respond to the different challenges 
involved in the process. They should help 
balance competing pressures faced by a 
national health research system, such as: basic 
vs. applied research; public vs. private 
research; health needs vs. political interests; 
national vs. international funding; public vs. 
private funding. These criteria should also 
respond to health needs reflected at different 
levels in the country. 


Looking at the reality and needs of the world’s 
lowest income countries, the current offering 
of tools and methods needs to be improved on. 
These countries require robust and 
inexpensive tools that can be effective in data 
poor environments. Situation analyses in these 
countries may have to rely on information 
which is relatively simple and affordable to 
obtain - for example, the use of mortality 
figures instead of disability adjusted life years 
(DALYs) in the measurement of health 
problems. 


A common feature of currently employed tools 
is that they use and build on past and current 
data on health status and a health situation in a 
country, rather than looking at future 
problems for which research is needed. The 
application of Foresight methodologies will 
bring a useful new perspective to the more 
traditional priority setting methods. They 
promote broader thinking and improve system 
preparedness - including its human and 
financial resources - to deal with various 
future scenarios. This perspective also looks at 
what needs to be developed now to address 
these future problems, it allows adjustments to 
research strategies and policies. It also helps 
identify the approach that is closest to the 
reality of that moment. 


Because requirements and opportunities for 
the choice of how to proceed with priority 
setting are dependent on a country specific 
situation at a moment in time, it 1s difficult to 
make general recommendations on ‘which tool 
is best’. Research managers will benefit from 
drawing on the experiences of colleagues in 
other countries to learn what methods and 
approaches have worked well (or not), the 
modifications and adjustments made and the 
results of other practical experiences. 


1.2.4 Equity orientation and 
legitimacy 

A priority setting process should help promote 
equity in health and development. To be 
effective, it is important to have agreement on 
the values and criteria that should influence 
health research priorities. The process should 
use fairness and legitimacy as key ethical 
elements. Legitimacy is achieved by ensuring 
the participation of the relevant stakeholders 
in the process. A useful framework for a fair 
process - Accountability for Reasonableness - is 
outlined by Daniels and Sabin (see annotated 
bibliography in section 2) and specifies four 
conditions for a fair process: relevance, 
publicity, revision and enforcement. Ensuring 
that the principles of fairness and legitimacy 
are followed requires capacity building and 
continuous improvement right through the 
process. 


1.2.5 Communication, dissemination 
and feedback of information 


An information and feedback strategy should 
be an integral part of any priority setting 
process and national research agenda. 
Continuous dissemination of research results 
and feedback to key players and beneficiaries 
of the process is crucial to get and retain the 
support from partners. Basic principles for 
effective communication are to use simple 
messages and understandable language, 
communicate frequently and include all 
groups that have been actively involved in 
defining the agenda. The communication and 
delivery of messages in ‘real time’ - as the 
process progresses - is crucial. 


Documenting the progress of the priority 
setting process helps increasing its 
transparency and credibility. Yet, this is rarely 


done, creating lack of clarity and causing loss 
of trust. 


In communicating the research agenda to 
community members, special care should be 
taken to explain how their concerns have been 
addressed in the national research agenda. In 
some situations, community priorities only 
have local relevance and may not appear in a 
national research agenda. In this case, a 
specific recommendation can be made to 
address these priorities at the local. level, 
through local institutions. Documenting these 
decisions and reporting them back to the 
community members are key actions to retain 
their interest and commitment during 
revisions of the agenda. 


Policy makers’ primary interest will be to 
know how the agenda (and the research to be 
conducted) will positively impact the health of 
the population and the decision making 
process. 


1.2.6 Scale and scope of priority 
setting activities: Starting 
small... What can be done now? 


As a component of the national health research 
system, priority setting needs - ideally - to 
build on comprehensive nationwide data and 
analysis. Gathering this depth of information 
may not be possible in the initial stages. A 
practical approach of incremental progress - 
“start small and build from there” - is a good 
option in many cases. Initial priority setting 
could focus on a region or a community, or on 
specific diseases or institutions. These small 
experiences will produce lessons to help 
develop a broad national agenda. 


For a single institution or research council, 
setting and implementing priorities is a 
relatively straightforward task. A ‘multiple 
entry point’ approach should also be 
considered - that looks at priority setting at the 
disease and institutional levels. A higher level 
of coordination is needed when several 
institutions are involved, or when these 
institutions are of different types - such as 
public sector, councils, NGO or private sector. 
Priority setting should be seen as an ongoing, 
iterative process in which the quality of data 
that is used to set priorities improves over 
time, and in which an increasing number of 


institutions and partners are involved. The 
extent to which priority setting covers a broad 
range of research areas in one or several 
sectors may vary from country to country. 
Furthermore, national priority setting must 
build on inputs and outputs at several levels 
(district, national and_ regional levels, 
institutional levels, etc). 


Examples from both the South and the North 
show that it is possible to link publicly funded 
applied and basic health research under a 
jointly organised priority setting and funding 
body (see examples from Philippines in section 
1.3.1 and the Netherlands in section 1.3.4). But it 
is not clear if there are instructive examples on 
how to include research funded by NGOs, the 
private sector and external donors into 
national priority agendas. 


1.2.7 ‘Investigator’ and ‘curiosity- 
driven’ research 


A research agenda should leave space for 
‘investigator’ and ‘curiosity-driven’ research. 
This approach suggests several advantages 
when improving country health research 
priorities. It facilitates the involvement of the 
research community in the research agenda 
setting process. It ensures links between the 
science and technology (aimed at promoting 
innovation and discovery) and health sectors 
(aimed at implementing more cost-effective 
interventions). It allows the development of 
research, within the national health research 
system, in areas that may not be seen as a 
priority at the moment of priority setting. And 
it gives access to international _ scientific 
developments. 


Leaving space for investigator and curiosity- 
driven research implies that, despite a good 
priority setting process and the use of good 
tools, the resulting research agenda will be 
imperfect. Scientists may identify problems 
that have not been picked up in the agenda 
setting process, and are encouraged to pursue 
their ideas if sufficient space is created for 
investigator-driven research. 


1.2.8 Translating priority health 
issues into priority health 
research issues 


A priority setting process helps define the 
health problems of a country. Health 
interventions in a country should be informed 
by evidence. Lack of evidence (as is the case in 
many developing countries) may be resolved 
through research. This includes developing 
more cost-effective interventions, identifying 
reasons for lack of use of the health system, or 
developing new technologies and innovations. 
It is important to make the distinction between 
health priorities and health research priorities, 
and to communicate and discuss the purpose 
of the priority setting process to all 
stakeholders involved. This ‘helps avoid 
frustration in community groups that may not 
see the health problems they initially defined 
reflected in the final health research agenda. 
On the other hand, priority setting of health 
problems also provides an opportunity to link 
the debate about health research priorities to 
more general health and development 
priorities, thus re-emphasising the link 
between research, health and development. 


1.2.9 Resources for health research 
priorities 

A clear view of the options and approaches for 
mobilising human, financial and institutional 
resources is crucial for putting an effective 
priority setting process into action. Pooling of 
funds between public funding bodies is one 
way of enhancing the chances of implementing 
national priorities (see examples of Philippines in 
section 1.3.1 and The Netherlands in section 1.3.4). 


The possibility of accessing funds from 
research funding organisations outside the 
national public sector remains a challenge. 
World Bank development grants often specify 
a proportion for research. How can these funds 
be channelled to health research? How can 
countries ensure that funds from other external 
research donors, including private industry, 
focus on supporting national priorities? How 
can a constructive dialogue be developed with 
external research funding organisations to 
convince them to support national priorities 


and the development of national health 
research systems? 


National health and research institutions need 
to develop their professional and_ technical 
capacity to set priorities and ito engage 
partners and lead the priority setting process. 
A country’s initial environmental scan should 
include an assessment of existing human 
resources. The priority setting process needs to 
assess whether the organisations’ and partners’ 
current skills are adequate to address the 
issues defined. Gaps in skills and expertise 
identified by the scan, then, become a priority 
area to address. The Foresight methodology 
further supports this by building the 
assessment of future skills into the national 
plan, and by specifying a training plan. 


1.2.10 Implementation, monitoring 


and evaluation 


Monitoring and evaluation are vital elements 
in priority setting, and should become 
standard practice, just as it is standard practice 
in health programs. This implies that budget 
allocation is needed to ensure monitoring, 
evaluation and follow-up of the research 
agenda. This is the basis for building a 
continuous and iterative research priority 
setting process. It should actively engage an 
increasing number of players to help improve 
management of the process over time. This 
cyclical approach ensures that the process is 
kept alive and up-to-date. Furthermore, there 
is a need for the systematic sharing of lessons 
learned in the monitoring and evaluation 
processes, including the development of both 
process and outcome indicators. 


1.2.11 Sustaining priority setting as a 
function of the national health 
research system 


In planning a priority setting process, research 
managers need to assess what is needed to 
make it work in a practical way. The grand 
vision and big picture of what the process 
should achieve in terms of health and equity 
for the country is necessary. But perhaps most 
important in the beginning are small practical 
steps that can show progress. 


Priority setting should be flexible, mapped out 
over the short, medium and long term, and 
subjected to regular review and _ reflection. 
When putting priority setting into action, a 
practical and realistic approach is needed. The 


overall view needs to be long term. But there 
will also have to be ‘quick wins’ - shorter 
practical steps along the way - to sustain 
motivation among participants. Addressing 
crises and political imperatives will require 
specific short-term objectives. Medium and 
longer term goals and useful milestones 
should be defined as a part of the plan. 


Taking a process perspective puts the 
emphasis on_ delivering a _ plan _ for 
implementation, with financial and human 
resources mapped out (or gaps identified) and 
including components for performance 
evaluation, capacity building and quality 
improvement. 


Special attention should be given to changes in 
government and administration to ensure that 
the set priorities are respected. Other elements 
on the time axis such as monitoring and 
evaluation and dissemination of information to 
key stakeholders, are activities that will keep 
the priority setting process alive. 


1.2.12 Mechanism for appeal 


Even with optimal preparation, use of suitable 
tools, and involvement of multiple 
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stakeholders, it is likely that i) some partners 
are not in agreement, or ii) that priorities 
change over time, sometimes at relatively short 
notice — for example to address new infectious 
diseases or newly defined health problems. 
Experience shows that if the priority-setting 
process has a space for negotiation and 
‘appeal’, it is much more likely to become a 
truly national agenda, one in which a much 
larger proportion of stakeholders can find 
themselves. An independent committee may 
be convened especially for this purpose. 


1.2.13 Advocacy 


Advocacy is important at both national and 
international levels - both in countries 
venturing into priority setting activities and in 
countries where priority setting structures and 
processes have been initiated. The rationale 
and purpose of priority setting has to be 
communicated and accepted by health 
research stakeholders at all levels to sustain 
priority setting as a function of the national 
health research system. Advocacy should 
extend beyond the health sector and also be 
directed to proponents of other sectors, which 
harbour important health determinants (e.g. 
water, sanitation, agriculture). 


1.3. Country experiences 


1.3.1 Philippines’ 


Environment 


In the early 1990s the Department of Health 
(DoH) in the Philippines took the lead in 
setting health research priorities. The Ministry 
of Science and Technology (S&T) - under 
which the Philippine Council for Health 
Research and Development (PCHRD) 
functions - set its own agenda. The financial 
contribution to health research from the S&T 
sector is larger than the contributions provided 
to health research from the DoH. This created a 
situation where priorities were only able to 
influence part of the health research carried 
out. 


To remedy this imbalance, the primary health 
research stakeholders in the Philippines - the 
DoH and the PCHRD - decided to take a more 
comprehensive systems approach, by bringing 
together the agendas and funding streams of 
the DoH and the PCHRD. This agreement was 
formalised in a memorandum of agreement 
signed between the main actors. It specifies the 
creation of a general fund for health research, 
and appoints PCHRD as the lead agent for the 
priority setting process. A law has been 
drafted (but has yet to be passed by 
Parliament) to formalise this agreement. While 
the new partners were willing to pool funds, 
the priority setting process had to be clear, and 
both sectors needed to be involved. 


Priority setting process 


The priority setting process applied a bottom- 
up approach, with five key steps: 


1. Division of the country into six zones with 
the National Capital Region as one zone. 
This was done to avoid dominance of the 
participants from the National Capital 


Region over participants from other 
regions. 
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2. Designation of convenors by zone tasked 


to oversee the priority setting process at 
regional and zone levels. 


3. Designation of region-based experts 


responsible for facilitating the writing of a 
situation analysis and conducting the 
region consultations to identify regional 
priorities. The situation analysis served as 
the benchmark and common set of criteria 
for workshop consultations to identify 
health and related problems from which 
research topics were identified and 
prioritised. 


4. Convening a zone assembly to bring 


together participants from the different 
regions of each zone to validate the 
consolidated zone report (prepared by the 
zone convenor) and arrive at a consensus 
and ranking of priorities. 


5. Convening a task force to formulate a set 
of national priorities based on the results 
of the regional and  zone-level 
consultations. 


The DoH and the PCHRD, with all key 
stakeholders, are represented and involved in 
the various steps of the process. 


This bottom-up process was started in 1999. 
Revisions to the agenda are made yearly. 
There is as yet no system in place to monitor 
the implementation and impact of the agenda. 
In addition, the feedback to and from all 
participants in the process needs to be 
strengthened. 


Lessons learned 


e Continuous communication and 
feedback. The priority setting process 
started at the regional level and ended 


with synthesising a list of priorities for the 
national level. 


Team = members highlighted the 
importance of providing feedback at zonal 
and regional levels, and communicating 
how and why certain priorities have been 


included in or excluded from the national 
agenda. 


Decentralised structure of the health 
system. The Philippines has a 
decentralised health and health care 
system, where local governments take 
budgetary decisions. This bottom-up 
experience made people feel empowered 
and involved in the decision making 
process. Some argued that the bottom-up 
approach could be best complemented 
with top down initiatives so that both 
approaches would capture what the top 
wants and what the bottom needs (Note: 
70% of funding for health research comes 
from central government). 


Using a _ bottom-up approach, and 
engaging local stakeholders in the process, 
helps advocate and raise awareness of the 
need for health research and can influence 
the budget that is made available to health 
research, also creating conditions for 
community to influence decisions at local 
level. 


Timing and frequency of the consultation 
process: This experience showed that 
research agenda setting is best done after 
the administration has set its plans and 
programmes and new initiatives have been 
identified. Reviews conducted to date have 
allowed the integration of new priorities 
(such as disaster management in response 
to natural disasters taking place), and the 
consideration of global developments and 
their relevance to the national health 
research situation. However, in general, 
the reviews do not show major changes in 
the priority issues on the agenda. 


In the Philippines, ministries need to 
redefine their priorities when a new 
president takes up office (every 6 years). 
Optimal timing is to set priorities at the 
start, and review these at mid-term and 
end-term. 


Expertise and commitment of 
stakeholders. It was a major challenge to 
find regional experts who were able to 
deliver the needed information in time. 
The quality of participation of stakeholders 
was also an issue, as not all participants 
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considered the process relevant. In 
addition, the private sector was not 
represented in the process and there is no 
mechanism in place to facilitate links 
between the public and private sectors. 


e Advocacy and independent advice. The 
involvement of independent regional or 
international actors helps support local 
stakeholders in increasing the political 
commitment for health research. COHRED 
was involved in this process in the 
Philippines. 


Further reading: 


e Alan B Feranil (2004) The Bottom Up Approach 
in Research Agenda Setting: Lessons from the 
Philippine Experience. Philippine Council on 
Health Research for Development. Paper 
presented at Forum 8, Global Forum for 
Health Research, Mexico 2004; downloaded 
on April 10, 2006 
from _http://www.globalforumhealth.org/ 
Forum8/Forum8-CDROM/Presenters.html 


e Department of Health in collaboration with 
Department of Science and Technology 
(2004). Proceedings of the 1st Philippine 
National Health Research Assembly, 2 June 
2004, Philippines. 


1.3.2 South Africa? 


Environment 


South Africa used the Essential National 
Health Research (ENHR) approach to set 
priorities in 1997. The present governing 
political party had adopted the ENHR strategy 
already during the period of exile. The 1996 
process came at a time when South Africa was 
going through major political developments 
and was ready for a change and for a focus on 


equity. 


The process followed the five-step approach 
recommended by the Ad Hoc Committee on 
Health Research Relating to Future 
Intervention Options. It called for information 
on the burden of disease, determining the lack 


———————— 
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of success to reduce the burden, trying to 
understand the contribution research can make 
in reducing the burden, and the development 
of ‘platforms’ to address the major issues - 
calling for a system approach that looks 
beyond disease. The agenda set in 1996 has 
been adopted by the government and accepted 
by the Department of Health (DoH) and by the 
Department of Science and Technology 
(DoS&T). This agenda is used by the Medical 
Research Council and by researchers in their 
grant applications. Implementation of the 
agenda was largely done by those involved in 
the priority setting process and committed to 
taking it forward. 


Priority setting process 


Following the ENHR priority setting process, 


the DoS&T? decided to do a Foresight exercise, 
looking at 12 sectors including health. The 
Foresight process used the ENHR priorities set 
in 1996, and added the concept of ‘multiple 
futures’. Where the ENHR process assumed 
one single future, the Foresight process 
assessed macro scenarios presenting multiple 
futures and the response of the S&T sector. The 
Foresight process identified critical questions 
and used the Delphi method to involve a 
broad group of people in the processes. 


Critical questions included the prioritisation of 
certain problems over others; the assessment of 
the need to create partnerships (international, 
private, etc.) to address each problem vs. the 
capacity of national players. Various 
implementation strategies were presented. The 
prioritisation of responses to questions was 
done using a common set of criteria at all 
levels of the consultation. 


The outcome of this process led to the 
development of several ‘roadmaps’. For 
example the discussion on HIV/AIDS vaccines 
moved beyond vaccine development to needs 
regarding national capacity to deal with the 
epidemic, public-private-partnerships, 
developments in other parts of the world and 
strategies to be developed if no vaccine would 
become available in the future. The Foresight 
process thus catalysed the development of 
strategies around biotechnology, drug 
development, health innovation and cost 
effectiveness of the health system. Most of the 
Foresight strategies developed have been 
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implemented, but no formal monitoring or 
evaluation has been done. 


Lesson learned 


The burden of disease approach is a useful 
starting point for priority setting on current 
issues. The Foresight approach identifies 
future issues or problems at an early stage and 
helps develop strategies to address them. 
These approaches provide complementary 
information and input to the priority setting 
process. 


Further reading 


Department of Health, Directorate Research 
Coordination and Management (1996). 
Proceedings of the First ENHR Congress on 
Priority Setting, November 14-15, South Africa 
e Department of Health (2001). Health 
Research Policy in South Africa. 


e Foresight method (See section 2.2). 
e Delphi method (See section 2.2). 


e http://www.dst.gov.za/ 


1.3.3 Brazil 


Environment 


The new Brazilian government, which started 
in January 2003, emphasised the central role of 
the national health authority in structuring 
national efforts in health research. It created 
the opportunity to outline a National Policy for 
Science, Technology and Innovation in Health. The 
policy is built on two main principles: to 
increase equity in the health system and to 


increase equity in health outcomes between 
groups. 


Priority setting process 


Directly linked to the science policy 
development, the Ministry of Health (MoH) 
initiated the priority setting process in 2003. 


3 Contribution by Antonia Angulo 


A group appointed by the National Health 
Council proposed 20 sub-agendas for health 
research. As a second step, research priorities 
for each sub-agenda were identified during 
national seminars, involving over 500 
researchers and policy makers, during the 24 
National Conference on Science, Technology 
and Innovation in Health held in 2004. 


During the preparatory phase, 307 cities and 24 
states organised local conferences, involving 
some 15,000 people in total. Some 360 
delegates from the health sector were 
appointed at local conferences, to attend the 
National Conference. Representatives from 
the education and science and _ technology 
sectors participated. The national policy was 
approved during the National Conference, 
together with three sub-agendas. Currently the 
national policy and priority agendas guide 
investments from the MoH for research and 
development, and partly from the Ministry of 
Science and Technology. From 2003 to 2005, 24 
calls for proposals were launched. As a result, 
3,962 research projects were submitted and 
1,300 financed. 


Lessons learned 


e The focus of the priority setting process in 
Brazil has, from the beginning, been on 
implementation, i.e. through the calls for 
proposals. However, the Ministry of Health 
funds only a small part of the health research 
conducted in Brazil. One way of influencing 
other streams of resources is the organisation 
of calls for proposals jointly with the 
Ministry of S&T. 


° The Ministry of Health currently distributes 
small amounts of funds to a large number of 
projects. But, maybe it is better to have few 
priorities and large programmes addressing 
these? Information is needed on the impact 
of the small grants on addressing the 
research priorities. Such information will 
assist in making funding decisions in future. 


° Another outstanding challenge is the 
linkage and response of large research 
institutions in Brazil to the national agenda. 
Some of these institutions have not yet 
defined their agendas, or have set their 
research agendas out of the context of the 
national agenda. Involvement and 
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commitment of these institutions towards 
the national agenda is needed. 


Further reading: 


¢ Reinaldo Guimaraes, Leonor Maria Pacheco 
Santos, Antonia Angulo-Tuesta, Suzanne 
Jacob Serruya. Setting and implementing a 
National Policy for Science, Technology and 
Innovation in Health: lessons from the 
Brazilian experience. To be published in 
Cuadernos de Satide Publica. 


e Ministry of Health (2005). Agenda Nacional 
de Prioridades de Pesquisa em Satide. Brasil. 
ISBN 85-334-0827-3 


Ministry of Health (2005). Politica Nacional 
de Ciéncia, Tecnologia e Inovacgao em Satde. 
Brasil. ISBN 85-334-0933-8 


e http://www. saude.gov.br/sctie/decit 


1.3.4 The Netherlands* 


Environment 


The Netherlands Organisation for Health 
Research and Development (ZonMW) is a 
national organisation that promotes quality 
and innovation in the field of health research 
and health care. ZonMW actively promotes 


knowledge transfer and implementation, 
ensuring that knowledge is exchanged 
between all relevant stakeholders. The 
majority of ZonMW’s _ requests for 


programmes come from the Ministry of 
Health, Welfare and Sport (VWS) and the 
Netherlands Organisation for Scientific 
Research (NWO). The Ministry’s main concern 
is to contribute to public health, including 
prevention and health care services. NWO is a 
non-governmental organisation concerned 
with fundamental and _ strategic research. 
Priority setting for the funding received 
through NWO (30% of ZonMW’s budget) is 
mainly based on criteria of quality and 
scientific relevance. Priority setting for the 
funding from the Ministry (70% of ZonMW’s 
budget) is mainly guided by the relevance for 
public health and health care and the 
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likelihood of implementation of research 


results. 


Priority setting process 


The priority setting process related to 
ministerial funding starts with a request from 
the Ministry to set a research programme to 
address a specific health problem. In response 
to this request, ZonMW establishes a 
Programme Committee. The Committee is 
independent, represents a wide range of 
expertise, includes patient organisations and 
user groups, and is usually chaired by a retired 
professional (to ensure maximum 
independence). 


The Programme Committee assesses whether 
the question raised by the Ministry is an 
urgent health priority, if research can 
contribute to addressing the problem, and if 
the national research system has the capacity 
to tackle the problem. The Committee may also 
use Foresight studies to assess the need for 
research on the health issue in the future. The 
Committee examines one health issue at the 
time and does not set priorities for the whole 
health system. 


If the Committee decides that a problem is 
relevant and can be addressed through 
research, it presents a programme proposal to 
the Ministry. Once approved by the Ministry, 
ZonMW issues a call for proposals. The 
proposals are assessed primarily on their 
relevance to the programme objectives, and on 
the likelihood of implementation of the results. 
Proposals must include a _ dissemination 
strategy and budget. ZonMW ensures follow 
up and monitoring of the implementation of 
results. 


15 


The total cycle from the identification of a 
question by the Ministry, to the completion of 
research that addresses the question, can take 8 
- 10 years. To maintain a continual cycle of 
priority identification, ZonMW maintains 
contact with a large group of stakeholders in 
the country. 


ZonMW has funds available to address issues 
that surface between the cycles of the larger 
programmes. User groups and patient groups 
are consulted as a part of this process and 
included in Programme Committees. In some 
situations, an advisory council that has done a 
priority setting exercise formulates a question 
through the Ministry. In this situation, 
ZonMW does not repeat the priority setting 
exercise. 


Lessons learned: 


° The link between implementation-oriented 
research (funded through the Ministry) and 
basic research (funded through NWO) is not 
continuous. Programs that fund both areas 
are not yet developed. Questions come from 
the Ministry or from NWO. There is a need 
to better link the health and S&T sectors. 


e ZonMW’s activities cover a vast amount of 
public funding for health research in the 
Netherlands. But they do not provide 
agenda setting for ongoing research in 
universities, nor do they work with the 
private sector. A challenge for the 
Netherlands remains to develop a national 
agenda that all stakeholders buy into. 


Further reading: 


e http://www.zonmw.nl 


1.4 Making priority setting work for health research: 


roles for COHRED 


The priority setting consultation discussed the 
following areas in which COHRED can play a 
key role: 


1.4.1 Advocacy and Communication 


COHRED has an important and essential role 
in advocating for priority setting for health 
research, in making the case for priority 
setting. The argumentation and evidence that 
priority setting can work to make research 
more responsive to health needs are essential 
to motivate all stakeholders in a country to 
engage actively in the process. Sharing of 
experiences between countries and showing 
that priority setting is possible (also in 
resource poor settings) is meeded. This 
international perspective will help increase 
local acceptance, and support research 
managers who are leading priority setting 
processes in their country. 


1.4.2 Monitoring and evaluation 


COHRED can play a role in the monitoring 
and evaluation (M&E) of national priority 
setting processes. As a technical partner it can 
support the development and integration of 
appropriate M&E components in the 
implementation of national priority research 
agendas (without engaging itself in the 
administration of these activities). This 
technical and independent expertise is useful 
in a sometimes politically sensitive process of 
priority setting. COHRED can play an 
advisory role throughout the process, and an 
advocacy role in ensuring that M&E is 
standard practice in any priority setting 
process. It can also help ensure that resources 
are allocated for this purpose. 
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1.4.3 Process and Methods 


It is crucial to ensure that the process followed 
for setting the health research priorities is 
right. It should be continuous and focus on 
action to implement the defined agenda. 
COHRED has a role in guiding the 
development of inclusive national priority 
setting processes that are action oriented, 
cyclical, and build on large stakeholder 
involvement. 


In addition, COHRED can make various 
methods, tools and approaches accessible, and 
be instrumental in the sharing of experiences 
applicable to various contexts. Information on 
‘what works where and when’ is useful for 
countries starting a priority setting process, or 
for countries revisiting their process and 
agenda. 


1.4.4 Knowledge Sharing: Facilitate 
further learning around priority 
setting 


This consultation was a first step in a 
continuous learning process around priority 
setting for national health research. COHRED 
and partners will use web-based interactions 
and face-to-face meetings to engage many 
more research managers and development 
partners to build on new experiences and 
recommendations from countries to improve 
priority setting. People and organisations 
interested in priority setting for research will 
be encouraged to contribute to this ‘learning 
spiral’ and to share their expertise. COHRED’s 
role as a facilitator of this learning process 
reinforces its advocacy and advisory roles. 


2. Overview of priority setting approaches and 


methods 


ee ee 


2.1 Annotated bibliography 


2.1.1 Rudan, I., El Arifeen, S., Black, 
R.E., New methodology _for 
systematic priority setting in 
global child health research. In 


Forum 9: Global Forum for Health 
Research. 2005. Mumbai. 


The objective of this paper is to review existing 
literature on priority setting, assess the strengths 
and weaknesses of suggested approaches and 
propose a systematic methodology to identify 
research priorities in global health and nutrition. 


The proposed priority-setting methodology is 
designed to have several levels of activities, 
systematically addressing all principles upon 
which it is based. 


Basic Principles in priority setting: 


- Legitimacy and fairness 


Trans-disciplinary approach 


Involving the stakeholders in the process 


Promoting health and development on the 
basis of equity and health maximisation for 


the greatest number of people with fixed 
level of investment 


- Realistic assessment of 
deliverability and 
proposed research 


affordability, 
sustainability of the 


- Realistic assessment of likelihood of research 
success in terms of both reaching the 


endpoint, and the endpoint being effective in 
reducing disease burden 


- Respecting the principles of economy in 
terms of cost of research and 


cost- 
effectiveness of intervention delivery 
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Simplicity, comprehensiveness and 
applicability of the methodology to a variety 
of priority setting issues in developing 
country setting at all population levels (local, 
national, regional or global) and for one or 
several diseases risk factors 


Constitution of two groups, a 
multidisciplinary technical working group 
and wider reference group. The technical 
working group’s tasks would be to: 1) drive, 
oversee and coordinate process, 2) gather 
and organise the evidence for reference 
group, 3) convene reference group for 
priority setting exercise, 4) calculate scores 
and weights, 5) polish the outcomes of 
several deliberations. The reference group 
would: 1) advice the technical group, 2) 
agree and/or modify the criteria tentatively 
proposed by the technical group, 3) do the 
actual priority setting based on the evidence 
and information provided by the technical 


group. 


Definition of broad dimensions to prioritise 
competing research options. This step 
requires the analysis of research options in 
relation to criteria such as their potential for 
disease burden reduction, their impact on 
equity, their likelihood of success in reaching 
the endpoint, their likelihood of being 
effective in disease burden reduction, their 
affordability, deliverability and 
sustainability in the population of interest. 


Listing of all the competing research options 
by the type of disease burden they would 
impact to inform the priority issues ranking 
process. The approach is based on a 3 steps 
process: 1) review of literature and 
subsequent brainstorming sessions with the 
reference group; 2) categorisation of 
competing options according to the type of 
disease burden that they affect; 3) 


assessment of prevalence of risk exposure in 
a population of interest. Risk assessment 
takes into consideration the estimation of 
relative risks, the burden of disease of 
interest, the cost of research and delivery per 
unit of population and the level of existing 
funding that is already invested in each 
research avenue. 


e Evaluation and improvement of process 
through feedback: 1) compare observed 
outcome to that expected (deliverability, 
achieved reduction in burden of disease, 
improvement in equity, likelihood of 
success, observed  cost-effectiveness); 2) 
integrate all observed indicators into a 
methodology as empirical obtained values; 
3) monitor, evaluate and improve the 
process from a fairness point of view (to 
what extent is the process actually consistent 
with the five conditions of accountability for 
reasonableness?). 


e Implementing recommended priorities 
requires establishing good links between the 
priority-setting working group, international 
and national donors and national policy- 
makers. 


2.1.2 Lomas, J., Fulop, N., Gagnon D,, 
Allen, P., On being a good 
listener: setting priorities for 


applied health services research. 
Milbank Q, 2003. 81(3): p. 363-88. 


This paper describes how the lessons learned 
from and the approaches to setting priorities 
for funding health services are translated into 
setting priorities for funding health services 
research. Two case studies (England and 
Canada) are used to describe a_ ‘listening 
model! for setting user-driven health services 


research priorities. 


The proposed ‘listening model’ for priority 
setting is a stepwise approach, based on the 
principle of linkage and exchange between 
research funders and researchers on the one 
hand and the research's potential users on the 


other 


° Stakeholders should represent potential 
users of research. The mix of stakeholders 
will depend on the particular function of the 
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research priority. The role of stakeholders 
should be determined in relation to the 
health system for which priorities are being 
set (type of research, political and 
organizational context). Involving system 
managers and policymakers allows linkage 
and exchange between researchers and 
decision makers throughout the research 
process and increases their sense of 
ownership. System managers and 
policymakers do have specific knowledge of 
the issues in the system and are the ones 
who will ultimately choose to apply or 
ignore the results from health services 
research. 


Two approaches can be used for identifying 
and assembling any data needed for 


consultation. The ‘environmental — scan’ 
approach allows validation of issues 
identified by stakeholders, as well as 


identification of health service priorities of 
other research funding agencies. The 
‘consultation’ approach generates ownership 
in the stakeholder priorities and offers the 
doers and users of research an opportunity 
for linkage and exchange to better 
understand each other’s constraints and 
practices. The consultation with stakeholders 
should lead to identification of issues likely 
to be a priority over the next 3 to 5 years. 
This longer period allows insulation from 
immediate political controversy as well as 
coordination of the timelines of the 
stakeholders’ community with the structure 
of the research process. 


Criteria that can be used to translate priority 
issues into priority research themes: 1) the 
issue is likely to be a high priority for at least 
3-5 years; 2) there is not large stock of 
existing relevant research in the area; 3) the 
issue is amendable to a significant number of 
feasible and _ generalisable research 
questions; 4) the research capacity exists to 
respond with high-quality research on this 
issue; 5) decision makers are receptive to 
research on this issue; 6) decision makers 
would be able to use research results on this 
issue; 7) the research would have potentially 
high impact relative to its cost. 


Priority ranking is affected by stakeholders’ 
view of the world. Managers consider 
priorities in terms of issues, policy-makers 1n 


term of interest groups, public in terms of 
problems, researchers in terms of disciplines 
or methodologies and clinicians in terms of 
diseases. Two approaches are suggested for 
ranking priority issues. 1) The “technical 
assessment approach” allows ranking of 
priority issues across potential clinical trial 
investments. This approach tends to hide 
under a series of assumptions many value 
judgments that may reflect those of the 
broader population of users and payers. On 
the other hand it requires the adoption of 
single clear objective to guide the exercise. 2) 
The “interpretive assessment approach” is 
best applied in agency wide assessments. It 
relies on the subjective judgments of 
participants expressed through structured 
exercises. This approach offers the possibility 
of dealing with multiple assumptions and 
objectives at the same time. 


2.1.3 OECD, Priority setting: Issues 
and recent trends (chapter 3), in 
Governance of public research, 


toward better practices. 2003, 
Organization for Economic Co- 
operation and Development. p. 61-75. 


This chapter describes priority setting as a 
strategic process to increase the return on 
public investments in research. It shows that 
governments use various institutional 
mechanisms for this: national science and 
technology plans, (de)centralised advisory 
bodies, foresight processes and_ public 
consultation. It further describes how priorities 
are reflected in research funding decisions, and 
how recent reforms reflect the changing 
balance between top-down and_ bottom-up 
approaches. 


e Involving diverse stakeholders should 
contribute to increase transparency and 
accountability and allow to better respond to 
societal needs. Target stakeholders could be: 
business and civil society involved in the 
central advisory council on science and 
technology, mutli-stakeholders involved in 
bodies that coordinate or fund research, 
scientific experts, policy, 
community representatives. 


business and 


The identification of priority issues can be 
achieved by using national science and 
technology plans, _ by accessing to 
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decentralised advisory bodies as well as by 
implementing public consultations and 
foresight processes. Foresight processes can 
be performed through methods such as 
future research prospecting and technology 
roadmaps. The Futur method implies the 
organisation of a forum for open dialogue 
between diverse stakeholders, to identify 
future priority fields of research. Research 
prospecting allows the identification of novel 
research topics and fields. Technology 
roadmaps involve a planning process based 
on the projected needs of tomorrow's 
markets. This methods helps companies 
identify, select and develop technology 
alternatives. 


Criteria for priority setting should respond 
to the different challenges involved in the 
process: balancing competing pressures 
(basic versus oriented research, core funding 
versus project funding, competition from 
increasing industry funding), institutional 
funding (rigidity of the research system, 
autonomy of research institutions, financing 
of high risk pre-competitive research), 
responding to emerging technologies and 
societal needs, promoting multidisciplinary 
research. 


2.1.4 Gibson, J.L., Martin, D.K., 
Singer, P.A., Setting priorities in 
health care organizations: 
criteria, processes, and 


parameters of success. BMC Health 
Serv Res, 2004. 4(1): p. 25. 


This paper summarises the lessons learned 
from workshops conducted for Board 
members and senior administrators at three 
Canadian academic health science centers who 
were seeking ethics advice on how to improve 
priority setting in their organisations. 


Workshop participants identified a number of 
preparatory steps that should be taken before 
priority setting can begin and additional 
elements that were important to improve 
quality and strengthen capacity for fair priority 
setting in their organisations over time. 


e Need of: 1) involving stakeholders in the 
priority setting process, 2) forming a 
multidisciplinary executive decision-making 


involves 


group, 3) clarifying and determining specific 
responsibilities of the board and _ senior 
management in relation to the priority 
setting process. 


An effective communication strategy should 
be developed to ensure transparent process 
and that stakeholders know and understand 
the scope and necessity of the priority setting 
decision-making, the degrees of freedom 
within priority setting and the particularities 
of the process (who will do what?, how?, 
why?). 


The following criteria should be considered 
when setting priorities in health care 
organisations: strategic fit, alignment with 
external directives, academic commitments, 
clinical impact, community need, 
partnerships, interdependency, and resource 
implications. Fairness is a key ethical goal of 
priority setting when health care resources 
are scarce. 


Quality improvement and_ capacity 
strengthening should be developed for fair 
priority setting. Process monitoring and 
formal evaluation strategies should be 
developed to ensure quality improvement 
and organisational learning. Process should 
be supported by leadership development 
and change management strategies to 
strengthen institutional capacity for priority 
decision making. 


2.1.5 COHRED, Priority setting for 
health research: lessons from 
developing countries. The 


Working Group on Priority Setting. 
Health Policy Plan, 2000. 15(2): p. 130-6. 


This paper proposes a strategy of priority 
setting, based on lessons learned from essential 
national health research (ENHR) approaches 
attempted in several developing countries. 


The proposed model aims at equity in health 
and development, it is demand-driven, and 
multi-dimensional inputs and 


multiple stakeholders. 


° Different approaches may be applied in the 
formation of the group in charge of the 
priority setting process. Possible approaches 
could be: 1) the organisation of a national 
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workshop followed by the formation of a 
task force to refine the research agenda, 2) 
formation of an_ inter-sectoral and 
multidisciplinary working group appointed 
by the Ministry of Health, 3) a research 
institute or university tasked to develop and 
propose processes for priority setting and a 
larger group of participants tasked to apply 


the proposed processes. Stakeholders 
involvement should be multilevel 
(communities, districts, sub-national, 


national) and multidimensional (quantitative 
and qualitative scientific input, social, 
economic political and management). 
Involving major stakeholders in priority 
setting fosters ownership of process and 
output, and facilitates shared responsibility 
and accountability in the implementation of 
the research agenda. Target stakeholders 
could be: researchers, decision makers at 
different levels, health service providers, 
communities, private sector, 
parliamentarians, potential donors, 
international agencies. 


The consultative group processes should be 
inclusive, participatory, interactive and 
iterative. Approaches could be: collection 
and analysis of health information, people’s 
consultations through focus-group 
discussions or interviews, consensus 
building on specific thematic areas, decision- 
making for determining and applying 
criteria for priority setting, translation of 
priority research areas into specific research 
programs and projects. 


Research priorities will depend on a two- 
step process: 1) selection of criteria for 
priority setting, 2) selection of research 
topics among identified priority problem 
areas. Some examples of criteria categories 
could be: magnitude and urgency of the 
problem; extent of previous research and the 
potential contribution of research in 
discovering, developing or evaluating new 
interventions; feasibility of carrying out the 
research in terms of technical, economic, 
political, socio-cultural and ethical aspects; 
expected impact of the research considering 
direct and indirect effects, short- and long- 
term benefits, implications on issues of 
affordability, efficacy, equity and coverage. 
The selection of the final criteria will depend 
on the purpose and level of action of the 


priority-setting exercise, on the availability 
of information related to the specific 
criterion and on the ability to define and 
measure the criteria in a common 
framework. 


Evaluation can be achieved through the 
assessment of indicators. Indicators could be: 
1) utilisation of the research agenda (How 
extensive was the implementation of the 
research priorities?, How much interest did 
the research agenda generate among 
stakeholders?); 2) involvement of multiple 
stakeholders (How many groups were 
involved and what were _ their 
contributions?); 3) equity in health (Do the 
research priorities address equity in health?, 
proportion of researchers that address health 
problems of the poor, shift of resource flows 
towards equity-targeted programs and the 
identified research priorities). 


2.1.6 NIH, Setting research priorities 
at the National Institute of 
Health, US National Institute of 
Health. 

Available from: 


www.nih.gov/about/ researchpriorities.htm 
(Accessed: February 13, 2006) 


This booklet describes the principles and 
mechanisms that guide the NIH (National 
Institutes of Health) in the continuous activity 
of managing its budget. 


¢ NIH involves stakeholders at different 
levels: general public, patients and _ their 
advocacy groups, institute staff, congress, 
scientist council members, ad hoc advisors, 
physicians and other health professionals, 
industry scientists, industry managers, 
professional societies, public members of 
advisory councils, boards of scientific 
counselors, president and administration, 
scientific review committees, scientists. 


NIH seeks opinion and counsel through 
methods like review groups; national 
advisory councils to review policy, with 
members from the public and from the 
medical and scientific community; advisory 
groups of outside experts to assess trans- 
NIH activities and to recommend budgetary 
and programmatic improvements; 
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consultations with federal agencies for 
budgetary and programmatic insight. 


Criteria determining NIH decisions are: 1) 
obligation to respond to public health needs 
as judged by the incidence, severity, and cost 
of specific disorders; 2) stringent review for 
scientific quality on all research proposals to 
return the maximum on public’s investment 
in medical research; 3) portfolio must be 
large and diverse (support research along 
broad, expanding frontier) because 
discoveries can not be predicted and 
opportunities that fresh discoveries may 
produce can not be anticipated, 4) continual 
support to the human capital and material 
assets of science by supporting: research 
training, acquisition of equipment and 
instruments, some limited construction 
projects, institution’s costs for enabling the 
research programs. 


2.1.7 Jackson, A. Strategic futures 
planning, suggestions for 
success, 2005. 


Available from: 

www.foresight.gov.uk/HORIZON_ SCANNING _C 
ENTRE/Toolkit/Toolkit.html 

(Accessed: February 13, 2006) 


This toolkit proposes the use of futures 
approach in the priority setting process. The 
approach helps building new networks, it 
creates a shared vision of how to move 
forward where a number of organisations have 
a stake in an issue, it highlights challenges and 
opportunities, tests robustness of policies and 
allows the optimal use of resources. 


e Methods that can be applied for the 
identification of priority issues: Horizon 
scanning, Delphi, Trend analysis, Driver 
analysis, Scenarios, Visioning, Technology 
roadmaps, System maps, Back-casting, 
Modeling, Simulation, Gaming, Data review, 
In-depth interview, Focus-group discussions, 
Consultative meetings, Round _ tables, 
Surveys, Field visits, Workshops, Seven 


questions, Issues trees, System maps, Review 
of areas of science. 


2.1.8 Musgrove, P., Public spending 
on health care: how are different 
criteria related? Health Policy, 1999. 
47(3): p. 207-23. 

This paper describes the relationship between 

nine different criteria considered to be relevant 

for decisions about public spending for health 
care. These include economic efficiency criteria 

(public goods, externalities, catastrophic cost, 

and ___ cost-effectiveness), ethical reasons 

(poverty, horizontal and vertical equity, and 

the rule of rescue), and political considerations 

(especially demands by the public). 


Criteria are hierarchically related. Sometimes 
two criteria will not be compatible but will 
conflict, forcing difficult choices--particularly 
between efficiency and equity. Properly 
thought-out choices of which health care 
interventions to finance with public funds 
therefore depend not only on looking at all 
these criteria, but also on treating them in the 
appropriate sequence and taking account of 
their possible inconsistencies. 


e Criteria for deciding on public spending 
should be based on economic efficiency, 
ethical reasons and political considerations. 


e The economic efficiency dimension takes 
into consideration: cost-effectiveness 
(relation between the cost of an intervention 
and the resulting health gain), public goods, 
externalities, and catastrophic cost. 


e Ethical reasons concern: poverty, horizontal 
equity (giving equal treatment to people 
with equal health problems, implying equal 
effectiveness), vertical equity (preferential 
treatment for people with worse problems), 
and the rule of rescue (grouping patients 
into: those whose lives can be saved by 
intervening, those who will die even if given 
treatment, those in between because their 
lives are not immediately threatened). 


° Political considerations relate to public 
demand (what the public thinks its money 


should be used for). 
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2.1.9 Fleurence, R.L,, Torgerson, D.J., 


Setting priorities for research 
Health Policy, 2004. 69(1): p. 1-10. 


This paper evaluates the way research is 
valued and assesses whether the 


corresponding priority setting method can 
meet the objectives of the health system. 


Different approaches that have been used in 
practice to value research and set priorities 
were reviewed. 


e The “subjective methods” approach bases 
most decisions on the scientific relevance as 
well as excellence of research proposals. The 
value put on research is subjective and 
dependent on the particular group that 
conducted the priority setting exercise. This 
type of approach is unlikely to lead to an 
allocation of resources that is consistent with 
the objectives of the health system. 


e The “valuating the burden of disease” 
approach values the ‘size’ of the disease with 
the assumption the higher the burden of the 
cost to society of the disease, the greater the 
need for research. Priorities in research are 
based on the relative contribution of diseases 
to the total burden and set by ranking 
diseases in terms of their cost to society. This 
approach assumes that the burden of disease 
rankings can be translated into the need for 
research. In the absence of a measure of the 
value of research, priorities set by burden of 
disease or cost of illness methods cannot lead 
to an appropriate allocation of resources that 
meets the objectives of the health system. 


e The “valuating the impact on clinical 
practice” approach values the impact of 
changes in clinical practice that occur as a 
result of the research taking place. This 
approach assumes that health benefits are 
dependent on the results of research and on 
changes in clinical practice that would occur 
as a consequence of the research results. 
Impact is measured by valuating the costs 
and benefits of conducting and 
implementing research. Fleurence et al. 
consider that such approach may not meet 
the objectives of health systems. Research 
money is more efficiently spent on 
interventions where uncertainty is great, 


Oe 
cee le 


rather than on conducting trials with the 
main objective of changing clinical practice. 


e The “valuating information” approach 
assumes that information provided by 
research can be measured and valued, and 
inform the decision to conduct research. The 
framework separates the decision to adopt a 
technology based on current information 
from the decision to conduct further 
research. This approach recognises the 
uncertainty inherent in the decision due to 
the uncertainty in the parameters that enter 
the decision. A quantitative measure of 
uncertainty is provided by the use of 
decision analytic models. 


2.1.10 Gericke, C.A., Kurowski, C., 
Ranson, M.K., Mills, A., 
Intervention complexity - a 
conceptual framework to 
inform priority-setting in 
health. Bull World Health Organ, 
2005. 83(4): p. 285-93. 


This paper describes a conceptual framework 
for the analysis of interventions according to 
the degree of technical complexity; this 
complements the notion of institutional 
capacity in considering the feasibility of 
implementing an intervention. 


The framework is illustrated using the 
examples of scaling up condom social 
marketing programs, and the DOTS strategy 
for tuberculosis control in highly resource- 
constrained countries. 


e “Intervention complexity” framework could 
be used to inform priority setting in health; 
as a tool for policy-makers, planners and 
program managers when considering the 


expansion of existing projects or the 
introduction of new interventions. 
e This approach categorises interventions 


according to their degree of complexity, 
identifies supply- and demand-side 
constraints, points to potential areas for the 
improvement of specific aspects of each 
intervention, provides overview of which 
human resource skill level is needed for 
which aspect of the intervention, assists in 
identifying bottlenecks, indicates where 
focus for future professional development 


ZS 


should lie and it can be used both to define 
what is feasible locally and to identify the 
best way to deliver an intervention. 


Main criteria considered in this approach are: 
intervention characteristics (basic product 
design, supplies, and equipment), delivery 
characteristics (facilities, human_resources, 
communication and transport), government 
capacity requirements (regulation/ legislation, 
management systems, and collaborative 
action), usage characteristics (ease of usage, 
pre-existing demand, black-market risk). 


2.1.11 Daniels, N., Sabin, J., Setting 
limits fairly: Can we learn to 
share medical resources? Oxford 


University Press, 2002, ISBN 0-19- 
514936-X 


The central idea of this book is that there is a 
lack of consensus on principles for allocating 
resources. In the absence of such consensus a 
fair priority setting process is needed for 
setting limits on health care. 


The authors propose four conditions for 
rationing, termed as "accountability for 
reasonableness": 


e Publicity. Decisions and the rationales for 
decisions such as coverage for new 
technologies or the contents of a drug 
formulary must be accessible to clinicians, 
patients and_ potential health plan 
subscribers — or citizens in a_ publicly 
administered system. 


Relevance. The grounds for such decisions 
must be ones that fair-minded people can 
agree are relevant to meeting health care 
needs fairly under conditions of reasonable 
resources. 


Appeals. There must be mechanisms to 
challenge and resolve limit-setting decisions 
and opportunities to revise and improve 


policies in the light of new evidence or 
argument. 


e Regulation. There must be some form of 
regulation to ensure that the other conditions 
are met. These regulations could come 


through governmental regulation, or be 
voluntary. 


The "accountability for reasonableness" seeks to educate the public about the need to 


approach speaks directly to the public set limits to health care. 
deliberation that is central to a democracy and 


2.2. Methods for priority setting 


METHOD DESCRIPTION APPLICATION Source 


Horizon Scanning e Looking across an area to 


identify future challenges 
and opportunities 
e It can range from 
systematic to more open 
exploration of an area 
e It can be near-term or 
look for issues at the 
limit of current thinking 


¢ Spot key issues before 
undertaking an in-depth 
analysis of interaction 

and social context 


“Strategic futures 
planning, 
suggestion for 
success” 
(www.foresight.go 
v.uk), by Andrew 
Jackson, March 
2005 


e Get and overview of 
what is happening in an 
area of science 


e Type of consultation 

e A two stage process 
starting with a 
questionnaire to seek 
initial views from a wide 
range of experts. 

e The responses are 
collated and sent out 
again to the contributors 
for comments 

e The experts are also 
asked to assess their 
relative level of 
competence in answering 

the questions 


CAUTION: it does not 
encourage interaction and 
it is very dependent upon 
experts finding the time to 
complete a series of 
questionnaires 


e Testing policy 
robustness and spotting 
developing problems 


Trend Analysis Extrapolating historical 
data forward 

More suited when 
looking at an issue that 
takes time to change, 
rather than an issue that 
is very responsive to 


immediate pressure 


CAUTION: its great 
weakness is that the future 

is rarely an extrapolation 

of the past 
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Testing policy robustness 
and spotting developing 
problems 


e Spotting the factors 
which are driving and 
shaping the trends and 
considering possible 
future interactions 
Suited to do a detailed 
study of interactions, 
looking for critical 
factors which are likely 
to shape the future 

e Suited to use as the basis 
for the development of 
scenarios, roadmaps or 
visions 


Driver Analysis 


Visioning Creating a rich picture of 
what the future might 
look like based on less 
rigour and more 
imagination 


Backcasting 


CAUTION: 
- If used for a detailed study 
the challenge is to quantify 
the relationship between 
the different drivers 
- If used for the development 
of scenarios, the challenge 
is to decide which of the 
many scenarios will be 
built from the information 
gathered 


Test the robustness of 
policies against a range 
of future challenges 
Spot the unexpected, 


Pictures of what the 
future might look like 
Internally consistent and 
built up from an 


assessment of how both potential 
trends and drivers might challenges and 
influence the present to opportunities 


create the future 
Usually 3 to 4 scenarios 
are produced 


Explore the context into 
which strategies and 
policies can be played 
out 


To increase the chances 
of a_ vision having 
success, it should have a 
ring of truth. The best 
way to achieve this is 
for the vision to reflect 
emerging patterns that 
will strike a chord with 
those hearing it 


In the broadest sense, 
roadmaps set out the 
steps to achieve a desired 
goal 

A technology roadmap 
often includes an 
assessment of the social 
drivers, science drivers, 
technologies and their 
applications 


Explore possible future 
products and the key 
pieces of science one 
would need to integrate 
to deliver those 
products 

Map out specific action 
one wants to take to 
deliver a new 
technology 


First producing an ideal 
future and then 
considering the steps one 
will need to take to 
increase the chances of 
achieving the desired 
outcome 


CAUTION: only possible if 
one has clear 
unambiguous aims 
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Modelling ¢ Good data is necessary to 


build and calibrate 


¢ To consider a complex 
issue where debate is 


models focused on people’s 

e They can be built when perceptions rather than 
examining the future of a evidence 
system 


CAUTION: one should be 

aware of the limitations of 
the figures produced by the 
applied model 


It is necessary to have an 
understanding of the 
factors that will affect the 
way the system will 
change over time 

¢ Models can provide 
metrics to help assess the 
relative impact of 
different options in more 
objective fashion 


Simulation e To communicate the 


complexity of decisions 
to a wide audience 

e To broaden perspectives 
of implementing 
policies 


Simulated scenarios 
where a computer model 
plays all the other parts 
and one as individual 
has a chance to see the 
effects of one’s decisions 
in the complexity 


Stakeholders are asked to 
assume that they find 
themselves in a scenario 
and are asked to decide 
how they would respond 


e For owners of policies to 
see how policies they 
are setting today may 
influence the future and 
how robust they will be 

in the long term 


“Investing in 
Health Research 
and 
Development’, 
Report of the Ad 
Hoc Committee on 
Health Research 
Relating to Future 
Intervention 
Options. WHO, 
Geneva, 1996 


Help decision-makers 
make rational choices in 
investment decisions so 
as to have the greatest 
reduction in the burden 
of disease for a given 
investment 


Ad Hoc Committee 
on Health Research 
Approach 


A five-step process 
aiming to allocate limited 
resources efficiently and 
effectively between a 
large number of possible 
research projects so as to 
have the largest possible 
impact on the health of 
the largest possible 
number of people 
e It analyses mostly 
biomedical determinants 
Cost-effectiveness 1s 
measured in terms of 
DALYs saved for a given 
cost 
DALYs: number of years 
of healthy life lost to each 
disease 


“The combined 
approach matrix: a 
priority setting tool 


e Bring together ina 
systematic framework 
all information related 


e A relation of the five-step 
process in priority setting 
(economic axis) with the 


Combined approach 
matrix 
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Advisory Committee 
on Health Research 
Approach 


Essential National 
Health Research 
Approach 


actors and factors 
(institutional axis) 
determining the health 
status of a population 

It helps organise, 
summarise and present 
all available information 
in one disease, risk 
factor, group or 
condition 

It facilitates comparisons 
between the likely cost- 
effectiveness of different 
types of interventions at 
different levels 


Multidisciplinary 
approach based on the 
analysis of various 
determinants: 
biomedical, economic, 
social, behavioural, etc. 
Analysis of health status 
and the recognition of 
deficits 

Identification of 
imperatives and 
opportunities for global 
health research 

Transfer of knowledge to 
programs which aim to 
reduce the health deficits 


Multidisciplinary and 
cross-sectoral approach 
Systematic analysis of 
health needs, societal and 
professional expectations 
Involves researchers, 
policy-makers, health 
care providers and 
community 
representatives 
Participatory and 
transparent process 
Facilitates partnership 
development 


EAE 


to a particular disease or 
risk factor 

Identify gaps in 
knowledge and future 
challenges 

Identify ‘common 
factors’ by looking 
across the diseases or 
risk factors 


Analyse the current 
world health and 
development conditions 
and problems 

Identify what is known 
and not known 

Identify what the 
Research Agenda refers 
to as research 
‘imperatives’, 
‘opportunities’ and 
ultimately ‘priorities’ 
Develop new 
knowledge, 
methodologies and 
approaches that 
contribute to problem 
solving and ultimately 
to better health 


Guide resource 
allocation and donor 
investments in health to 
areas of highest 
priorities 

Address the issue of 
equity 

Direct attention to the 
most vulnerable groups 
of the population 
Reinforce the links 
between research, action 


and policy 


for health 
research”, edited 
by Abdul Ghaffar, 
Andres de 
Francisco and 
Stephen Matlin. 
Global Forum for 
Health Research, 
June 2004 


“ A research policy 
agenda for science 
and technology, to 
support global 
health 
development”. The 
advisory 
committee on 
health research. 
WHO, 1997 


“Priority setting 
for health research: 
lessons from 
developing 
countries”. The 
working group on 
priority setting, 
COHRED. In 
Health Policy Plan, 
2000. Vol 15 (2), 
pages 130-136. 


Annex 1: Agenda 


COHRED Think Tank meeting 
Priority Setting in Research for Health 


Geneva, 20 and 21 February, 2006 
Venue: Conference Centre Varembé (Geneva) 


Monday February 20, 2006 (9am - 
5pm) 


The aim of the consultation is to increase our 
understanding of how priority setting for 
health research can contribute to health and 
development at the country level. The 
consultation is the starting point of an ongoing 
learning process, aimed at developing an 
approach towards priority setting that can be 
continuously adapted when new experiences 
become available. 


The first day of the consultation will focus 
around the question ‘How to develop a 
national priority research agenda for health?’ 
During this day we will take stock of past 
experiences and look at the usefulness of 
priority setting, and the best processes and 
best methods to set priorities. 


9 am - 12 am: 
e Introductions: 


o Carel Isselmuiden: Welcome and 
general aims of the consultation 


o Sylvia de Haan: Outline of the two 
days, process that will be followed & 
key areas to be addressed in priority 
setting for research for health 


* Discussion on aims of the meeting and the 
main issues to be discussed 


* Why set health research priorities? What is 
the usefulness of priority setting? 


General brainstorming session identifying 
the main reasons for defining a priority 
research agenda, looking at economic, 
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political, ethical and health reasons, and 
considering the overall research system 
and key values of research for health. 
Other issues that need to be considered 
include: the definition used for research; 
the link between national and international 
research agenda’s. 


Which process is needed to set national 


health research priorities? Examples of best 
practice? 


Who should lead the process, who else 
should be involved, and when should 
people and organisations be involved? 
What are the mechanisms and agreements 
needed to guide and coordinate the 
process? Which preparatory work is 
needed to define priorities: decisions re 
information needed, methods to be used, 
criteria and ranking to be applied? What 
should the focus of priority setting be 
(disease oriented versus more system 
oriented)? What is the time perspective 
needed for priority setting (short versus 
long term perspective)? What type of 
research should be prioritised? 


Which methods can be used? 


Review of existing methods, criteria and 
ranking of criteria and the experiences that 
participants have in applying these. Is it 
possible to recommend the use of certain 
methods? What are advantages and 
disadvantages of methods? 


Tuesday February 21, 2006 


9am - 4pm 


The second day of the consultation we will 
look forward and address the question ‘What's 
next?’ What is needed to make priority setting 
work? A more detailed agenda for day two 
will follow from the discussions of the first 
day, but some of the questions that will be 
addressed include: 


* How can priority setting for health 
research influence public debate around 
health and development issues? 


e Should priority be given to conduct 
research in the areas with a highest chance 
of success? 
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How to use national priorities to influence 
international priorities? 


How to monitor and evaluate progress 
with the implementation of a research 


agenda? 
How to keep a priority agenda alive? 


How can the priority setting process 
contribute to the development of a 
national health research system? 


How can COHRED best support national 
priority setting? (Issues of brokerage and 
enabling) 
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